Human Data: From Gathering to Sharing

Department of Cognitive Science, Macquarie University
omidreza.ghasemi@hdr.mq.edu.au

Data Collection

Experiment Softwares

Psychopy

PsychoPy is an free cross-platform package allowing you run a wide range of in the behavioral sciences (neuroscience, psychology, psychophysics, linguistics…). (Source: Psychopy website)

Psychopy: Task Builder And The Actual Task

• Build your first PsychoPy experiment Link 

• Build the Posner cueing task Link 

• A series of tutorials for building a memory task Link 

• Create basic components using the coder view link

• Book: Building Experiments in PsychoPy (by Jonathan Peirce and Michael MacAskill) link

• Book: Programming Visual Illusions for Everyone (by Marco Bertamini) link

• Paper: PsychoPy2: Experiments in behavior made easy link

• Get stuck? Ask your questions here

Online Platforms

Qualtrics

Qualtrics: Builder View And The Actual Survey

• Demo: A simple numeracy task link

• A nice introduction to making a qualtrics survey link

• Qualtrics Beginner Tutorial part 1 and part 2

• And a more advanced Qualtrics tutorial link

Psychopy (PsychoJS)/Pavlovia

PsychoJS is a JavaScript library that makes it possible to run neuroscience, psychology, and psychophysics experiments in a browser. It is the online counterpart of the PsychoPy Python library (Source: PsychoJs repository). Pavlovia.org is a site created by the PsychoPy team to make it easy to run studies online, host your experiments, share studies with other scientists, and version control your work. (Source: Psychopy website)

• Demo: The Posner task link

• Upload your Psychopy experiment to Pavlovia link

• Running your experiment on Pavlovia.org link

• Paper: The timing mega-study: comparing a range of experiment generators, both lab-based and online link

• Get stuck? Ask your questions here

Gorilla

The Gorilla Experiment Builder is a cloud based research platform that allows researchers and students to quickly and easily create and deploy behavioural (reaction-time) experiments online (Source: Gorilla website)

Gorilla: Task Builder And The Actual Task

• Demo: The dot-probe task link

• Getting started with Gorilla link

• Gorilla support page link

• Gorilla Demos and Examples List link

• Paper: Gorilla in our midst: An online behavioral experiment builder link

JsPsych

jsPsych is a JavaScript library for running behavioral experiments in a web browser. The library provides a flexible framework for building a wide range of laboratory-like experiments that can be run online (Source: jsPsych website)

• Create a simple Reaction Time Task link

• A more detailed tutorial on jsPsych and Google app engine link

• Online linguistics experiments using jsPsych (Na-Young Ry, 2018) link

• Paper: jsPsych: A JavaScript library for creating behavioral experiments in a Web browser link

Jaysire

The goal of jaysire is to provide a method for writing behavioural experiments in R that can be deployed through a web browser. The package relies on the jsPsych library by Josh de Leeuw (GitHub page) to create the experiments, and is structured so that functions in jaysire use the same argument names as the corresponding jsPsych functions. (Source: jaysire repository)

• Demo: A choice reaction time task link

• Create an online experiment using Jaysire part 1 part 2 part 3

• Behavioural experiments with jaysire link

REDCap

REDCap is a secure web application for building and managing online surveys and databases. While REDCap can be used to collect virtually any type of data, it is specifically geared to support online or offline data capture for research studies and operations (Source: project-redcap.org)

Empirica

Empirica is an open-source JavaScript framework for running multiplayer interactive experiments and games in the browser. It was created to make it easy to develop and iterate on sophisticated designs in a statistically sound manner, and offers a unique combination of power, flexibility, and speed. (Source: empirica.ly)

• Getting started with Empirica link

• A series of tutorials for building a simple interactive task Link 

• Using Empirica for high-throughput virtual lab experiments (an interesting experiment) Link 

Data Cleaning and Analysis

R

• Book: R for data science link

• Book: ggplot2: Elegant Graphics for Data Analysis link

• Book: Fundamentals of Data Visualization link

• Book: Learning statistics with R: A tutorial for psychology students and other beginners link

• Book: R for Psychology link

• Book: A Language, not a Letter: Learning Statistics in R link

• Book: Foundations of Statistics with R link

• Book: Statistical Rethinking with brms, ggplot2, and the tidyverse link

• Course: Data Science: Foundations using R link

• Example: Doing an Analysis of Variance (ANOVA) in R link

Python

• Book: Python for Everybody: Exploring Data Using Python 3 link

• Course; Python for Everybody Specialization link

• Data Analysis in Python with Pandas link

• Data Visualization in Python using Matplotlib link

• Example: Doing an Analysis of Variance (ANOVA) in Python link

Reproducible Research

Until quite recently, making research open required a great deal of extra work for researchers and was less likely to be done. Now, with available tools, the benefits of openness can be a by-product of time-saving efficiencies, because tools that reduce data headaches also result in science that is more transparent, reproducible, collaborative and freely accessible to others.  (Lowndes et al., 2017)

Source: Our path to better science in less time using open data science tools

• FAIR Principles:

  • Findable

  • Accessible

  • Interoperable

  • Reusable

• The FAIR Guiding Principles for scientific data management and stewardship link

Folder Structure

• Setting up an Organised Folder Structure for Research Projects link

• Transition your workflow, keep your sanity link

• Organising Your Data link

• 23 (research data) Things link

• Project-oriented workflow link

Git

Source: phdcomics.com

• Happy Git and GitHub for the useR link

• The Turing Way: Version Control link

• Reproducible workflow and version control with Git and Github link

Rmarkdown and Jupyter Notebooks

R Markdown provides an authoring framework for data science. You can use a single R Markdown file to both save and execute code [and] generate high quality reports that can be shared with an audience R Markdown documents are fully reproducible and support dozens of static and dynamic output formats. (Source: rstudio.com)

• How does an Rmarkdown file look like? link

• How does a Jupyter notebook file look like? link

• Rmarkdown - Introduction and Basics link

• Reproducible analysis using R, Rmarkdown, and the tidyverse link

• Using Jupyter notebooks for interactive analyses and reproducible science link

Data Sharing

Open Science Framework (OSF)

OSF is a free, open-source Web tool designed to help researchers collaboratively manage, store, and share their research process and files related to their research. (Source: Soderberg, 2018)

You can preregister a study, upload a preprint, and connect your Github repository, Goggle Drive, or oneDrive to your project.

• A sample project link

• An OSF beginner tutorial link

• Paper: Using OSF to Share Data: A Step-by-Step Guide: link

Zenodo

Zenodo is a general-purpose open-access repository developed under the European OpenAIRE program and operated by CERN. It allows researchers to deposit data sets, research software, reports, and any other research related digital artifacts. For each submission, a persistent digital object identifier (DOI) is minted, which makes the stored items easily citeable. (Source: Wikipedia)

• A sample project link

• How to upload files to Zenodo file